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Learning Content, 

Not Just Completing a Course^ 

A growing number of educators and policymakers have begun focusing on the goal of preparing all or nearly all high 
school students for college and other advanced postsecondary learning opportunities. To meet this goal, states are 
adopting policies to encourage nearly all students, not just those seeking entry into competitive four-year colleges, to 
take a full core curriculum of college-preparatory courses in high school.^Yet it is much easier to give students credit 
for a course labeled "Algebra II" than it is to ensure that those students actually learn algebra. So how do educators and 
policymakers know that students are actually learning the content implied by the course titles? 


Course Credit Inflation 

As enrollment in college-preparatory courses expands 
to include a large percentage of the student population, 
troubling evidence has accumulated that many students 
— particularly disadvantaged ones — are receiving credit 
for the courses but are not learning the content. For exam- 
ple, in one state with a longitudinal student data system: 

l> 63 percent of low-income, 61 percent of African Ameri- 
can and 59 percent of Hispanic students who graduated 
under the state's new recommended college-preparatory 
graduation plan in 2000 needed remediation in one or 
more subjects when they enrolled in the state's public 
higher education institutions. The corresponding per- 
centages for white and non-low-income students were 
27 percent and 33 percent.® 

I> 58 percent of low-income, 67 percent of African 
American and 57 percent of Hispanic students who 


' The thesis of this report is taken from C. Dougherty, L. Mellor and S. Jian, Orange juice or 
Orange Drink? Ensuring that "Advanced Courses" Live Up to Their Labels, NCEA, 2006, www. 
just4kids.org/en/research_policy/college_career_readiness/. 

^ By 2008, 19 states included at least four years of English, three years of mathematics, 
three years of science and three years of social studies in their high school graduation 
requirements (www.achieve.org/files/50-state-2008-final02-25-08.pdf). Two states have 
adopted policies making a college readiness core course curriculum the default graduation 
plan for all high school students (www.act.org/news/data/07/states.html). Twenty-four 
states have "State Scholars" programs that encourage students to take college-preparatory 
courses in high school (www.wiche.edu/Statescholars/). 


received course credit for Algebra I in 1999 failed the 
corresponding end-of-course exam. For white and non- 
low-income students, the corresponding figures were 
35 percent and 39 percent. 

I> In the 2002 high school graduating cohort, the pass- 
ing rate for low-income students who took academic 
Advanced Placement (AP) courses and the correspond- 
ing AP exams was less than 25 percent. The passing 
rate for non-low-income students who took both AP 
courses and exams was 58 percent. 

The course-taking patterns of students who took the ACT 
exam provide national evidence that courses completed 
may overstate content learned.^ For example, of students 
who reported that their high school mathematics curricu- 
lum consisted of Algebra I, Geometry and Algebra II, only 
26 percent met the ACT college readiness benchmark for 
college algebra. Only with additional mathematics course- 
work did the percentage rise above 50 percent.® 


^E-mail communication from James Dilling, Texas Higher Education Coordinating Board, 
March 3, 2005. This and the following two examples are taken from Dougherty, Mellor and 
Jian, 2006. An earlier analysis by Lopez, 2000, using data on 1997 high school graduates 
found that about half of students receiving "Advanced" or "Advanced with Honors" 
diplomas needed remediation in one or more subjects when they entered Texas public 
higher education institutions. 

^This report uses data from ACT, Inc. due to availability. Similar analyses could be done 
with SAT data. 

^ ACT, Inc., Rigor at Risk: Reaffirming Quality in the High School Core Curriculum, 2007. 
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An analogy may be made to truth-in-labeling laws in 
business. A company selling an orange-colored beverage 
under the label "orange juice" can get in legal trouble if 
the beverage contains little or no actual juice. But there are 
no consequences for giving students credit for Algebra 11 
even if they have learned little algebra. In some cases, 
the problem is the lack of a standard definition of what 
content should be covered in an Algebra 11 course; in 
other cases, districts and states lack measures of whether 
the defined content has been taught and learned; and in 
still other cases, students receive credit for courses even 


though available measures indicate that they have not 
learned the content implied by the course titles.® 

Like the concept of grade inflation, giving students credit 
for courses when they have not learned the content may 
be labeled "course credit inflation." In the case of grade 
inflation, the knowledge and skill level of the median 
student receiving an A declines over time. In the case of 
course credit inflation, the level of content mastery by the 
median student receiving credit for a course with a given 
title declines over time.^ 


*The problem of course titles not reflecting course content, let alone what students are 
learning, has received attention from national organizations. The College Board has 
announced an initiative to review the content of courses taught under the AP label. ACT 
has developed a system to audit the content of high school courses and has written a 
report with The Education Trust on the content of model advanced high school courses. 

^See Dougherty, Mellor and Jian, 2006. 
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Addressing Content Mastery with the 
10 Essential Elements 


The 10 essential elements of a statewide longitudinal student data system make addressing guestions about course 
completion, course credit inflation and college readiness possible. 


1. A statewide student identifier is needed to link indi- 
vidual students' course-taking data with their scores 
on state tests and SAT and ACT exams and with infor- 
mation on their success in higher education. 

2. Student-level enrollment and demographic data are 
needed to see whether the problem of course credit 
inflation is worse for low-income and minority stu- 
dents than for non-low-income and white students 
and whether the problem shows up mainly in certain 
schools and districts. 

3. The ability to match student-level test records across 
years can be used to assess whether the students who 
are getting course credit when they haven't mastered 
the content are the same students who entered the 
class poorly prepared at the beginning of the year. 

4. Information on untested students can be used to see 
whether these students tend to be the ones who were 
less well prepared at the beginning of the year. 

5. Matching students with their teachers is critical for 
identifying classrooms in which course credit infla- 
tion is or is not a problem. In addition, educators and 
researchers may want to identify teachers who appear 
to be effective at working with students who are aca- 
demically behind. 

6. Student-level transcript (course completion and 
course grade) information, when connected to end-of- 
course exam data on the same subject, can be used to 
tell whether students who receive credit for a course 


appear to have mastered the course content. The 
information on student grades can be used to check for 
grade inflation. 

7. College readiness test data (SAT and ACT exam 
results) can be used to assess whether students 
completing a sequence of college-preparatory courses 
appear to be college ready. For example, a student who 
completed mathematics courses through Algebra II 
should be able to score at or above the college readiness 
benchmark on the ACT Mathematics exam or receive 
an analogous score on the mathematics section of the 
SAT Reasoning Test.^ 

8. Student-level graduation and dropout data can be 
used to see whether students completing various 
course sequences are graduating. In addition, gradu- 
ation data often contain information on the course 
sequences ("graduation plans") that students com- 
pleted to earn their diplomas. 

9. The ability to match student records between the 
P-12 and higher education systems is necessary to 
see whether students who complete various course 
sequences in high school succeed when they get to col- 
lege. For example, students who complete mathematics 
courses through Algebra II shouldn't need remediation 
in mathematics when they get to college. 

10. A state data audit system assessing data quality, 
validity and reliability is necessary to ensure that 
trends in course completion and course credit inflation 
are being evaluated using accurate data. 
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® ACT and the College Board have collaborated on comparing scores of students who take 
both exams to identify what SAT Mathematics score best corresponds to a given score on 
the ACT Mathematics exam. 
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Using Longitudinal Data To Address Key 
Questions about Course Completion 
and College Readiness 


As states develop longitudinal student data systems, educators and policymakers in those states should be able to 
answer questions such as the following: 


Question One: To what extent are students who 
receive credit in college-preparatory courses actually 
learning the content implied by the course titles? 

Answering this question requires the state to administer 
an end-of-course exam covering the specific content of the 
course and link student-level results from this exam with 
transcript data indicating which students earned credit for 
the course. For example, one state linked individual stu- 
dents' course credit and grade information on Algebra I 
with the same students' Algebra I end-of-course exam 
results. The state found that the majority of low-income 
and minority students receiving course credit for Algebra I 
failed the state end-of-course exam on the same subject.^ 

Moreover, collecting information on course grades makes 
looking at grade inflation possible. In the Algebra I study 
above, the state looked at average Algebra I end-of-course 
exam scores for students earning course grades of 60, 

70, 80, 90 or 100. Although students from all groups with 
course grades of 90 or 100 averaged better than a passing 
score of 1500 on the exam, minority students with a course 
grade of 80 — a comfortable B-minus — averaged a failing 
score on the exam. Thus, the data indicated that grade 
inflation was more pronounced for minority students 
(Table 1). 


Table 1 

Average Algebra I End-of-Course Test Scores for Given Algebra I 
Course Grades 

Course Grade 


Group 

60 

70 

80 

90 

100 

All Students 

1365 

1396 

1483 

1594 

1728 

African American 

1348 

1379 

1438 

1569 

1659 

Hispanic 

1368 

1385 

1447 

1536 

1645 

White 

1372 

1416 

1515 

1616 

1751 


Source: Texas Education Agency, 2000 
Failing scores on end-of-course test are shaded. 

Question Two: To what extent are students who com- 
plete a core sequence of college-preparatory courses 
prepared for college-level academic work? 

This question calls for matching data on the courses that 
students completed with evidence that the same students 
were well prepared for college. Preparation for college, 
in turn, can be assessed by results on college readiness 
tests taken in high school, academic placement tests taken 
when the students get to college, and students' success in 
staying in college and receiving a degree or credential. 
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^This result also is mentioned on page 1 of this resource guide. 
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In the example mentioned on page 1, the graduation data 
provided information on course sequences completed, 
academic placement tests reported on whether each 
student needed remediation in college prior to taking 
credit-bearing work, and demographic data were used to 
disaggregate the students by income and ethnicity. 

Question Three: What is the relationship between 
students' academic achievement levels prior to enrolling 
in the courses and their success in mastering the course 
content? 

By matching the same students' prior year state test 
results with their current year score on end-of-course 
exams, it is possible to identify what level of prior perfor- 
mance is predictive of student success in each course. An 
example of this approach is illustrated in Figure 1, which 
matches students' scores on the 8th grade state math- 
ematics test with the same students' scores on the state's 
Algebra I end-of-course exam. The figure shows that a 
student just barely meeting the state's passing standard 
of 70 on the 8th grade mathematics exam was unlikely 
to pass the Algebra I exam in the following year. This 
finding indicates that many students who passed the state 
test were likely to require extra help in their algebra class. 
School systems that are forewarned by this kind of infor- 
mation are more likely to plan for the necessary assistance 
for students. 

Question Four: Which schools and classrooms are doing 
the best job of teaching the content of specific courses, con- 
trolling for their students' prior academic preparation? 


Figure 1 

9th Grade Algebra I Exam Passing Rates in 2000 by Results 
on 8th Grade Mathematics TAAS Exam in 1999 

State passing Proficiency 
standard on TAAS standard on TAAS 


g 100% 



8th grade TAAS Mathematics Texas Learning Index (TLI) score 
Source: NCEA analysis, 2000 

classrooms, allowing for students' level of prior prepara- 
tion. For example, in some classrooms, even well-prepared 
students may be having trouble with Algebra I. In other 
schools and classrooms, well-prepared students may be 
learning algebra while poorly prepared students are not. 
Still other Algebra I teachers may be enjoying unusual 
success with poorly prepared students. Distinguishing 
these three types of cases entails matching students to 
their teachers as well as matching prior and current year 
test records for fhe same students. 


If students at a certain grade level aren't learning the 
content that they should be, it is important to be able to 
identify whether the problem is concentrated in certain 


5 


Data Quality Campaign 


October 2008 



Recommendations 


School systems and local and state policymakers must monitor whether students who enroll in and complete college- 
preparatory courses are in fact well prepared to do college-level work.'® This monitoring can be done most efficiently 
on a statewide basis so that each school district does not have to conduct its own separate research project. To make 
this analysis possible, states should: 

> Develop a longitudinal student data system with the 
10 essential elements, each of which can contribute to 
understanding students' levels of college readiness in 
the ways described in this resource guide. 

> Develop end-of-course exams for core high school 
courses. Because much of the development work on 
these exams already has been done, they do not need 
to be created from scratch. For example, ACT, Inc., 
has created a nationwide set of end-of-course exams 
as part of its QualityCore program, and Achieve is 
working with a consortium of sfates on an Algebra 11 
end-of-course exam. 

> Once the necessary data are in place, begin the analysis 
— or contract with outside researchers to conduct the 
analysis — to answer the four questions listed in this 
resource guide. 


’“For many students, college-level work will include technical training in community 
colleges and other venues to qualify for skilled careers. 
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In general, putting all or nearly all high school students 
into college-preparatory high school courses will accom- 
plish the desired goal of preparing students for college 
only if adequate steps are taken to prepare students in 
advance for success in the courses and to offer additional 
support to students who enter high school with gaps in 
their academic preparation. With a longitudinal data sys- 
tem, states can monitor student academic results over time 
to see if fhese things are happening. 
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